具有丰富注释的高质量结构化数据是处理道路场景的智能车辆系统中的关键组件。但是，数据策展和注释需要大量投资并产生低多样性的情况。最近对合成数据的兴趣日益增长，提出了有关此类系统改进范围的问题，以及产生大量和变化的模拟数据所需的手动工作量。这项工作提出了一条合成数据生成管道，该管道利用现有数据集（如Nuscenes）来解决模拟数据集中存在的困难和域间隙。我们表明，使用现有数据集的注释和视觉提示，我们可以促进自动化的多模式数据生成，模仿具有高保真性的真实场景属性，以及以物理意义的方式使样本多样化的机制。我们通过提供定性和定量实验，并通过使用真实和合成数据来证明MIOU指标的改进，以实现CityScapes和Kitti-Step数据集的语义分割。所有相关代码和数据均在GitHub（https://github.com/shubham1810/trove_toolkit）上发布。

Monitoring water is a complex task due to its dynamic nature, added pollutants, and land build-up. The availability of high-resolu-tion data by Sentinel-2 multispectral products makes implementing remote sensing applications feasible. However, overutilizing or underutilizing multispectral bands of the product can lead to inferior performance. In this work, we compare the performances of ten out of the thirteen bands available in a Sentinel-2 product for water segmentation using eight machine learning algorithms. We find that the shortwave infrared bands (B11 and B12) are the most superior for segmenting water bodies. B11 achieves an overall accuracy of $71\%$ while B12 achieves $69\%$ across all algorithms on the test site. We also find that the Support Vector Machine (SVM) algorithm is the most favourable for single-band water segmentation. The SVM achieves an overall accuracy of $69\%$ across the tested bands over the given test site. Finally, to demonstrate the effectiveness of choosing the right amount of data, we use only B11 reflectance data to train an artificial neural network, BandNet. Even with a basic architecture, BandNet is proportionate to known architectures for semantic and water segmentation, achieving a $92.47$ mIOU on the test site. BandNet requires only a fraction of the time and resources to train and run inference, making it suitable to be deployed on web applications to run and monitor water bodies in localized regions. Our codebase is available at https://github.com/IamShubhamGupto/BandNet.

In this paper, we discuss an imitation learning based method for reducing the calibration error for a mixed reality system consisting of a vision sensor and a projector. Unlike a head mounted display, in this setup, augmented information is available to a human subject via the projection of a scene into the real world. Inherently, the camera and projector need to be calibrated as a stereo setup to project accurate information in 3D space. Previous calibration processes require multiple recording and parameter tuning steps to achieve the desired calibration, which is usually time consuming process. In order to avoid such tedious calibration, we train a CNN model to iteratively correct the extrinsic offset given a QR code and a projected pattern. We discuss the overall system setup, data collection for training, and results of the auto-correction model.

Language-conditioned policies allow robots to interpret and execute human instructions. Learning such policies requires a substantial investment with regards to time and compute resources. Still, the resulting controllers are highly device-specific and cannot easily be transferred to a robot with different morphology, capability, appearance or dynamics. In this paper, we propose a sample-efficient approach for training language-conditioned manipulation policies that allows for rapid transfer across different types of robots. By introducing a novel method, namely Hierarchical Modularity, and adopting supervised attention across multiple sub-modules, we bridge the divide between modular and end-to-end learning and enable the reuse of functional building blocks. In both simulated and real world robot manipulation experiments, we demonstrate that our method outperforms the current state-of-the-art methods and can transfer policies across 4 different robots in a sample-efficient manner. Finally, we show that the functionality of learned sub-modules is maintained beyond the training process and can be used to introspect the robot decision-making process. Code is available at https://github.com/ir-lab/ModAttn.

We propose SparseFusion, a sparse view 3D reconstruction approach that unifies recent advances in neural rendering and probabilistic image generation. Existing approaches typically build on neural rendering with re-projected features but fail to generate unseen regions or handle uncertainty under large viewpoint changes. Alternate methods treat this as a (probabilistic) 2D synthesis task, and while they can generate plausible 2D images, they do not infer a consistent underlying 3D. However, we find that this trade-off between 3D consistency and probabilistic image generation does not need to exist. In fact, we show that geometric consistency and generative inference can be complementary in a mode-seeking behavior. By distilling a 3D consistent scene representation from a view-conditioned latent diffusion model, we are able to recover a plausible 3D representation whose renderings are both accurate and realistic. We evaluate our approach across 51 categories in the CO3D dataset and show that it outperforms existing methods, in both distortion and perception metrics, for sparse-view novel view synthesis.

The people in the world who are hearing impaired face many obstacles in communication and require an interpreter to comprehend what a person is saying. There has been constant scientific research and the existing models lack the ability to make accurate predictions. So we propose a deep learning model trained on ASL i.e. American Sign Language which will take actions in the form of ASL as input and translate it into text. To achieve the translation a Convolution Neural Network model and a transfer learning model based on the VGG16 architecture are used. There has been an improvement in accuracy from 94% of CNN to 98.7% of Transfer Learning, an improvement of 5%. An application with the deep learning model integrated has also been built.

Recent video+language datasets cover domains where the interaction is highly structured, such as instructional videos, or where the interaction is scripted, such as TV shows. Both of these properties can lead to spurious cues to be exploited by models rather than learning to ground language. In this paper, we present GrOunded footbAlL commentaries (GOAL), a novel dataset of football (or `soccer') highlights videos with transcribed live commentaries in English. As the course of a game is unpredictable, so are commentaries, which makes them a unique resource to investigate dynamic language grounding. We also provide state-of-the-art baselines for the following tasks: frame reordering, moment retrieval, live commentary retrieval and play-by-play live commentary generation. Results show that SOTA models perform reasonably well in most tasks. We discuss the implications of these results and suggest new tasks for which GOAL can be used. Our codebase is available at: https://gitlab.com/grounded-sport-convai/goal-baselines.

Industries must follow government rules and regulations around the world to classify products when assessing duties and taxes for international shipment. Harmonized System (HS) is the most standardized numerical method of classifying traded products among industry classification systems. A hierarchical ensemble model comprising of Bert- transformer, NER, distance-based approaches, and knowledge-graphs have been developed to address scalability, coverage, ability to capture nuances, automation and auditing requirements when classifying unknown text-descriptions as per HS method.

在图像分类中，在检测分布（OOD）数据时发生了许多发展。但是，大多数OOD检测方法是在一组标准数据集上评估的，该数据集与培训数据任意不同。没有明确的定义``好的''ood数据集。此外，最先进的OOD检测方法已经在这些标准基准上取得了几乎完美的结果。在本文中，我们定义了2类OOD数据使用与分布（ID）数据的感知/视觉和语义相似性的微妙概念。我们将附近的OOD样本定义为感知上相似但语义上与ID样本的不同，并将样本转移为视觉上不同但在语义上与ID相似的点数据。然后，我们提出了一个基于GAN的框架，用于从这两个类别中生成OOD样品，给定一个ID数据集。通过有关MNIST，CIFAR-10/100和Imagenet的广泛实验，我们表明A）在常规基准上表现出色的ART OOD检测方法对我们提出的基准测试的稳健性明显较小。 N基准测试，反之亦然，因此表明甚至可能不需要单独的OOD集来可靠地评估OOD检测中的性能。

通过查找图像可能不满意的图像来捕获对象检测器的错误行为，这一兴趣很长。在实际应用（例如自动驾驶）中，对于表征除了简单的检测性能要求之外的潜在失败也至关重要。例如，与远处未遗漏的汽车检测相比，错过对靠近自我车辆的行人的侦查通常需要更仔细的检查。在测试时间预测这种潜在失败的问题在文献和基于检测不确定性的传统方法中被忽略了，因为它们对这种错误的细粒度表征不可知。在这项工作中，我们建议将查找“硬”图像作为基于查询的硬图像检索任务的问题进行重新制定，其中查询是“硬度”的特定定义，并提供了一种简单而直观的方法，可以解决此任务大型查询家庭。我们的方法完全是事后的，不需要地面真相注释，独立于检测器的选择，并且依赖于有效的蒙特卡洛估计，该估计使用简单的随机模型代替地面真相。我们通过实验表明，它可以成功地应用于各种查询中，它可以可靠地识别给定检测器的硬图像，而无需任何标记的数据。我们使用广泛使用的视网膜，更快的RCNN，Mask-RCNN和CASCADE MASK-RCNN对象检测器提供有关排名和分类任务的结果。